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Abstract 



P^^ I In many spin glass models, due to the symmetry between sites, any limiting joint 

r^ • distribution of spins under the annealed Gibbs measure admits the Aldous-Hoover 

C^ , representation encoded by a function a : [0, 1]^ — t- { — 1,+1} and one can think of this 

function as a generic functional order parameter of the model. In a class of diluted 
models and in the Sherrington-Kirkpatrick model, we introduce novel perturbations of 
the Hamiltonians that yield certain invariance and self-consistency equations for this 
^ \ generic functional order parameter and we use these invariance properties to obtain 

-vi ' representations for the free energy in terms of a. In the setting of the Sherrington- 

t~^ ■ Kirkpatrick model the self-consistency equations imply that the joint distribution of 

^N I spins is determined by the joint distributions of the overlaps and we give an explicit 

^O ' formula for a under the Parisi ultrametricity hypothesis. In addition, we discuss some 

^^ ' connections with the Ghirlanda-Guerra identities and stochastic stability. 



Key words: mean-field spin glass models, functional order parameter, free energy. 



1 Introduction. 

In various mean-field spin glass models one considers a random Hamiltonian Hm{(t) indexed 
by spin configurations cr G S^r = { — 1,+!}^ and defines the corresponding Gibbs measure 
Gtv as a random probability measure on S^v given by 

GN{cT) = ^eM-HN{(T)) (1.1) 

ZjN 

where the normalizing factor Zjsf is called the partition function. Let (cr')i>i be an i.i.d. 
sequence of replicas from measure Gtv- Let hm denote the joint distribution of the array 
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of all spins on all replicas, (cr')i<i<Af,i<;, under the annealed product Gibbs measure KG%°° 
which means that for any choice of signs a' G { — 1,+1} and for any n > 1, 

^^({a^ = a( : 1 < i < A^, 1 < / < n}) = EGf^Ha^ = 4 : 1 < i < N,l < I < n}). (1.2) 

Usually, this distribution has the following two symmetries. First of all, it is obviously in- 
variant under the permutation of finitely many replica indices /. Moreover, in most spin glass 
models /ijv is also invariant under the permutation of coordinates i G {1, . . . , A^} since the 
distribution of Hn{(t) is often symmetric under the permutation of coordinates of cr, and 
this invariance of ^^ is called symmetry between sites. Let us think of ^n as a distribution 
on ((t') for all ?, / > 1 simply by setting a' = for i > N. Convergence of measures (/Ujv) 
will always be understood in the sense of convergence of finite dimensional distributions. In 
most models it seems hard to prove convergence of {hn) and for this reason we will consider 
a family Ai of all possible limits over subsequences of (/iAr). For example, in the setting of 
the Sherrington-Kirkpatrick model the answer to a weaker question of convergence of the 
distribution of one overlap is only known in a special case when all p-spin terms are present 
in the Hamiltonian (see [26]) and the proof is based on the validity of the Parisi formula 
for the free energy proved in [25]. Whenever we have symmetry between sites, any limiting 
distribution /i G A^ will be invariant under the permutations of both coordinates i and / 
and the representation result of Aldous [2] and Hoover [14] (see also [4]) for these so-called 
exchangeable arrays states that there exists a measurable function a^ : [0, 1]'^ — ?> M. such that 
the distribution fi coincides with the distribution of the array (s') given by 

4 = af,{w,ui,Vi,x^) (1.3) 

where random variables w, (ui), (fj), (xj^/) are i.i.d. uniform on [0, 1]. This function a^ is 
defined uniquely up to some measure-preserving transformations (Theorem 2.1 in [15]) so 
we can identify the distribution fi of array (s') with the function o"^. Since we only consider 
the case when spins and thus cr^ take values in {— 1,+1}, the distribution n is completely 
encoded in the function 

d-f,{w, u, v) = E^a^{w, u, v, x) (1.4) 

where E^, is the expectation in x only and we can think of this last coordinate as a dummy 
variable that generates a Bernoulli r.v. with expectation afj,{w,u,v). However, keeping in 
mind that a function of three variables a^ encodes the distribution of the array (1.3), for 
convenience of notations we will sometimes not identify a Bernoulli distribution with its 
expectation (especially, in the diluted models) and work with the function (T^{w,u,v,x). 

An ultimate goal would be to describe the set of possible limits /i G A^ and their 
representations o"^. In the setting of the Sherrington-Kirkpatrick model we will give such 
explicit description of all limits that satisfy the Parisi ultrametricity hypothesis for the 
overlaps which will be given by some specific realizations of the Ruelle probability cascades 
that encode not only the joint distribution of all overlaps but of all multi-overlaps as well. 
However, we will begin with a more modest goal. In the setting of the diluted models and 
the Sherrington-Kirkpatrick model, we will first narrow down possible limits Ai to some 
well-defined classes of distributions Aiinv that will be described via invariance and self- 
consistency equations for a^. Then, using these equations, we will prove representations for 



the free energy 

FM = j^mogZ^ (1.5) 

in the thermodynamic hmit A^ — )■ oo in terms of cr^ in Aiinv The proof of the invariance 
properties will be based on some standard cavity computations; however, justification of 
these computations will rely on certain properties of convergence of measures hn- In each 
model we will introduce a novel perturbation of the Hamiltonian that will force the sequence 
fiN to satisfy these properties, and the ideas behind these perturbations will constitute the 
main technical contribution of the paper. 

One can think of a function a^ (or a^) as a generic functional order parameter of the 
model and it is easy to see that information encoded by a^ is equivalent to the limiting joint 
distribution of all multi-overlaps 

Ri„...,u = N-' Yl ^^•••^'" (1-6) 

l<i<Af 

for all n > 1 and all /i, . . . , /„ > 1 under fi^. Indeed, by expanding the powers of (1.6) in 
terms of products of spins, in the limit one can express the joint moments of multi-overlaps 
in terms of the joint moments of spins and vice versa. By comparing these moments, the 
asymptotic joint distribution of (1.6) over a subsequence of fj,N converging to /x coincides 
with the joint distribution of 

Rh,-,iu =K^{w,ui„v) . . .a{w,ui„,v) (1.7) 

for a = o"^, for all n > 1 and all /i, . . . ,/„ > 1, where E„ is the expectation in the last 
coordinate v only. In the setting of the Sherrington-Kirkpatrick model we will show that, 
due to the Gaussian nature of the Hamiltonian, the self-consistency properties oi a = a^ for 
fj, G Aiinv imply that the joint distribution of all multi-overlaps (1.7), and thus measure /x, 
is completely determined by the joint distribution of the overlaps 

R^i, = Et, a-{w, ui, v)(t{w, ui', v) (1.8) 

for all /, /' > 1. We will give an explicit description of any a^ that satisfies the Ghirlanda- 
Guerra identities and Parisi ultrametricity hypothesis for the overlaps which states that 
i?^3 > min(i?f^2! -^fs) with probability one. This can be expressed in terms of a^ by saying 
that for all w G [0, 1] the family of functions v — )■ a^{w,u,v) parametrized by m G [0, 1] is 
ultrametric in >C^([0, l],dv). To illustrate main new ideas we will start with the case of the 
diluted models where many technical details will be simpler. 

1.1 Diluted models. 

We will consider the following class of diluted models as in [19]. Let p > 2 be an even integer 
and let a > 0. Consider a random function 6 : { — 1,+1}*' — > M and a sequence {dk)k>i 
of independent copies of 6. Consider an i.i.d. sequence of indices {ii^k)i,k>i with uniform 



distribution on {1, . . . , A^} and let 7r(Q;A^) be a Poisson r.v. with mean aN. Let us define the 
Hamiltonian Hj^{a) on Sjv by 

-H^{a)= J2 ^'^K.,- ••,%..)• (1-9) 

k<Tr(aN) 

Clearly, any such model has symmetry between sites. We will make the following assumptions 
on the random function 6. We assume that there exists a random function / : { — 1, +1} — t- M 
(i.e. /(a) = /' + f'cr for some random (/', /")) such that 

exp6'((Ti,...,o-p) = a(l + &/i(o-i) . . . /p(o-p)) (1.10) 

where fi, . . . , fp are independent copies of /, 6 is a r.v. independent oi fi, . . . , fp that satisfies 
the condition 

Vn>l E(-6)" > (1.11) 

and a is an arbitrary r.v. such that E| loga| < oo. Finally, we assume that 

|6/i(ai).../p(ap)|<la.s. (1.12) 

and 6 satisfies some mild integrability conditions 

— oo < Emin^(o"i, . . . , Cp), Emax^(o'i, . . . , ap) < +oo. (1-13) 

Two well-known models in this class of models are the p-spin and i^-sat models. 

Example 1. (p-spin model) Consider /3 > and a symmetric r.v. J. The p-spin model 
corresponds to the choice of 

9{ai,...,ap) = (3Jai...ap. 

(1.10) holds with a = ch(/3J), b = th(/3J) and f{a) = a and condition (1.11) holds since we 
assume that the distribution of J is symmetric. (1.13) holds if E| J| < oo. 

Example 2. {K-saX model) Consider /3 > and a sequence of i.i.d. Bernoulli r.v. {Ji)i>i 
with F{Ji = ±1) = 1/2. The i^-sat model corresponds to 

1<P 

(1.10) holds with a = 1, 6 = e"^ - 1 and fi{ai) = (1 + Ji(ti)/2 and (1.11) holds since 6 < 0. 

D 

It is well-known that under the above conditions the sequence A^-F/v is super-additive 
and, therefore, the limit of F^ exists (see, for example, [9]). If we knew that (/Ujv) has a 
unique limit, i.e. Ai = {/i}, then computing the limit of the free energy in terms of cr^ in 
(1.3) would be rather straightforward as will become clear in Section 2.2. However, since we 
do not know how to prove that (hn) converges, this will create some obstacles. Moreover, if 
ifJ'Nk) converges to /x over some subsequence (A^^) we do not know how to show that (fiNk+n) 
converges to the same limit for a fixed shift n > 1, even though we can show that it does 



converge simply by treating n of the coordinates as cavity coordinates. Even if we knew 
that /iTv converges, we would still like to have some description of what the limit looks like. 
To overcome these obstacles, we will utilize the idea of adding a "small" perturbation to 
the Hamiltonian (1.9) that will not affect the limit of the free energy but at the same time 
ensure that (yU,Arj.+„) and (/iTVfe) converge to the same limit. In some sense, this is similar to 
the idea of adding p-spin perturbation terms in the Sherrington-Kirkpatrick model to force 
the overlap distribution to satisfy the Ghirlanda-Guerra identities ([12], see also [10]). The 
perturbation for diluted models will be defined as follows. 

Consider a sequence (ctv) such that Cjv — > oo, Cj\f/N — )■ and [cat+i — cj\f\ — )■ 0. Consider 
an i.i.d. sequence of indices {ij^k,i)j,k,i>i with uniform distribution on {1, . . . ,N}, let 7r(cAr) 
be a Poisson r.v. with mean cn, (iii^ap)) be i.i.d. Poisson with mean ap and {9k,i) be a 
sequence of i.i.d. copies of 6. All these random variables are assumed to be independent of 
each other and of everything else. Whenever we introduce a new random variable, by default 
it is assumed to be independent of all other random variables. Let us define the perturbation 
Hamiltonian H^{(t) on Sjv by 

-Hl{a)= ^ logAv^exp ^ ^^X^. ^n.^.n • • • , ^Vw) (l-^^) 

where Av^ will denote uniform average over e G { — 1,+1} as well as replicas (ei) below. Let 
us redefine the Hamiltonian in (1.9) by 

-H,,icT)= J2 Okia,,,,...,a.^^,)-H^^ia) (1.15) 

k<'rr{aN) 

and from now on we assume that (/iat) and Ai are defined for this perturbed Hamiltonian. 
Obviously, condition (1.13) implies that the perturbation term does not affect the limit of free 
energy since c^ = o{N). The benefits of adding this perturbation term will first appear in 
Lemma 3 below where it will be shown that thanks to this term (fiNk) ^i-nd (uNk+n) converge 
to the same limit for any fixed shift n > 1. Another important consequence will appear 
in Theorem 1 below where the perturbation will force the limiting distributions fi E Ai to 
satisfy some important invariance properties that will play crucial role in the proof of the 
representation for the free energy in Theorem 2. 

Let us introduce some notations. We will usually work with a^ for a fixed distribution 
/i G A^ so for simplicity of notations we will omit subscript /i and simply write a. Let 
{vii,...,i„), (a:ii,...,j„) be i.i.d. sequences uniform on [0, 1] for n > 1 and ii, . . . ,in > 1 and let 

Siu...,in = <^{w,u,Vi,^,„,i„,Xi^„„,iJ. (1.16) 

The role of multi-indices (ii, . . . , ■?„) will be simply to select various subsets of array (1.3) with 
disjoint coordinate indices i without worrying about how to enumerate them. Let (^ii,...,j„) 
be the copies of random function 6 independent over different sets of indices. In addition, let 
v,x,6 be independent copies of the above sequences and let 

.,Xi,,...,i„). (1.17) 



Notice that we keep the same w and u in both s and s. Throughout the paper let us denote 
by vr(A) Poisson random variables which will always be independent from all other random 
variables and from each other. For example, if we write 71(0;) and 7r(/3), we assume them to 
be independent even if a = /3. Let {ttj{X)) be independent copies of these r.v. for j > 1. Let 

^j(^) = 2^ ^fc,i(^, Si,j,fc, • • • ,Sp-i,j,fc) (1-18) 

k<ni(pa) 

for i > 1 and e G { — 1, +1} and let 

Bi = 2_^ dk,i{^i,i,k-,- ■ ■ -iSp^i^k)- (1-19) 

k<-Ki((p-i)a) 

We will express invariance and self-consistency properties of distributions ^ E M. in terms 
equations for the joint moments of arbitrary subset of spins in the array (1.3). Take arbitrary 
n,m,q,r > 1 such that n < m. For each / < q consider an arbitrary subset C/ C {1, . . . , m} 
and let 

Cl=Cin{l,...,n}, Cf = Cin{n + l,...,m}. (1.20) 

Let E' denote the expectation in u and in sequences x and x and let 

Ui =E'Ave Yl ejexp^Aj(ej) JJ s^exp ^4(si,fc, • • • , Sp,/,) (1.21) 

ieC'l i<n ieCf k<r 

and 

V = E'Av, exp J2 M^'i) exp ^ 4(si,fe, • • • , Sp^k)- (1-22) 

i<n fc<r 

Then the following holds. 

Theorem 1 For any limiting distribution /i G A^ and a = a^ we have 

Enn^'=EnE'n-^=E%r^- (1-23) 

l<q ieCi l<q i<^Ci 

We will say a few words about various interpretations of (1.23) below but first let us describe 
the promised representation for the free energy. Let Aimv denote the set of distributions of 
exchangeable arrays generated by functions a : [0,1]'' — )■ {— 1,+1} as in (1.3) that satisfy 
invariance equations (1.23). Theorem 1 proves that A^ C A^j„„. Let 

-4(e) = ^ 6'fc(£, si,fc,...,Sp_i,fc) 

fc<7r(pa) 

foreG{-l,+l}, 

B = 2_^ ^fe(si,fc, • • • ,Sp,fc) 

/c<7r((p— l)a) 

and let 

V{fi) = log2 + ElogE'Av£expA(£) -ElogE'exp5. (1.24) 

The following representation holds. 
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Theorem 2 We have, 



lim Fn = inf V{fi) = inf V{fi). (1.25) 

Af— )-00 /^gAI H&Minv 



One can simplify the last term in (1-24) since we will show at the end of Section 2.3 that 

ElogE'exp5 = {p- l)aElogE'exp^(si,...,Sp) (1.26) 

for /i G J^inv To better understand (1.23) let us describe several special cases. Let us define 

A, = logAv,expA,(£). (1.27) 

First, if we set r = and let sets Ci be such that Ci C {n + 1, . . . , m} for all / < g then 
(1.23) becomes 

On the other hand, if we set n = then (1.23) becomes 

E ng n ^. - e"'^' '"' "'^^ " ""P ^'^- '•"''•■ ■ ■ ■ ■ '"-' , (L29) 

II ,:^^, (E'expE,<,9i(3i,..^..,3„))' 

These equations can be interpreted as the invariance of the distribution of (s') under various 
changes of density and they will both play an important role in the proof of Theorem 2. 
Another consequence of (1.23) are the following self-consistency equations for the distribution 
of spins. Let us set r = and n = m. Let 

^ _ Avs£expAi{e) 



Av^expA^e) 
Then (1.23) becomes 

eTTe' TT 5. = E ^'^'^^^^-^^''f'''P^-^"^\ (1.30) 

■'■■'■ ■'■■'■ E'expV.. Ai^ ^ ^ 

This means that the distribution of spins (s') coincides with the distribution of "new" spins 
(Sj ' ) under a certain change of density. Even though we can not say more about the role 
(1.30) might play in the diluted models, its analogue in the Sherrington-Kirkpatrick model 
will play a very important role in proving that the joint overlap distribution under /i de- 
termines fi and in constructing the explicit formula for a under the Parisi ultrametricity 
hypothesis. 

It will become clear from the arguments below that, in essence, the representation (1.25) 
is the analogue of the Aizenman-Sims-Starr scheme in the Sherrington-Kirkpatrick model [1] 
with one cavity coordinate. Previous applications of this scheme, for example in [1], [9] or [16], 
only worked when the number of cavity coordinates goes to infinity, since considering one 



cavity coordinate in general yields only a lower bound on the free energy. This lower bound 
expressed in terms of the generic functional order parameter a^ will be proved in Section 2.2. 
Then the main new ideas of the paper - the roles played by the perturbation Hamiltonian 
(1.14) and the consequent invariance in (1.23) - will help us justify that this lower bound 
is exact and, moreover, represent it via a well-defined family Aimv First, following the 
arguments in [11], [19], in Section 2.3 we will prove a corresponding Franz-Leone type upper 
bound which will depend on an arbitrary function a that defines an exchangeable array as 
in (1.3). For a general a, this upper bound will depend on N. However, we will show that for 
a^ for /x G Aiinv the invariance of Theorem 1 implies that the upper bound is independent 
of N and matches the lower bound. This is the main point where the invariance properties 
will come into play. The same ideas will work in the Sherrington-Kirkpatrick model with the 
appropriate choice of the perturbation Hamiltonian. 

1.2 The Sherrington-Kirkpatrick model. 

Let us consider mixed p-spin Sherrington-Kirkpatrick Hamiltonian 

-H^{(t) = -Y,PpHnA^) (1-31) 

p>i 

where 

-Hn,p{(t) = (p_iw2 J2 9ii,...,i,(^h ■■■^i,, (1-32) 



l<ii,...,ip<N 



the sum is over p = 1 and even p > 2 and {gi^^...^ij,) are standard Gaussian independent for 
all p > 1 and all (ii, . . . , ip). The covariance of (1.31) is given by 

¥.HN{a-^)HN{o-^) = N^{Ri,2) (1.33) 

where ^(x) = X]p>i f^p^^- Let us start by introducing the analogue of the perturbation 
Hamiltonian (1.14) for the Sherrington-Kirkpatrick model. Consider independent Gaussian 
processes G^'(cr) and Gg{cr) on Sjv = { — 1,-|-1}^ with covariances 

EG^,ia')G^,ia^) = e'(i?i,2), EGoi(T')Gei(T^) = ^(i?i,2) (1-34) 

where 9{x) = x^'{x) — ^{x) and let G^'^fc(cr) and Ge^k^cr) be their independent copies for 
k>l. For {cn) as above, let us add the following perturbation to the Hamiltonian (1.31), 

-Hlj{a)= Y, logchG'^,,,(^)+ Yl Ge,k{(^), (1.35) 

fc<7r(cjv) A;<7r'(cjv) 

where 7r(cjv) and 7r'(cAf) are independent Poisson random variables with means cjy. Clearly, 
this Hamiltonian does not affect the limit of the free energy since cat = o{N). We will see that 
this choice of perturbation ensures the same nice properties of convergence as the properties 
in Section 2.1 in the setting of the diluted models. As a consequence, we will get the following 



analogue of the invariance of Theorem 1. Given a measurable function a : [0, 1]^ — ?> [—1, 1], for 
any w G [0, 1] let g^i{(j{w^u^ ■)) be a Gaussian process indexed by -u G [0, 1] with covariance 

GoY{g^>{a{w, u, ■)),g^>{a{w, u', •))) = ^'{K^iw, u, v)a{w, u\ v)) (1.36) 

and gglalwjU, ■)) be a Gaussian process independent of g^i{(y{w,u, ■)) with covariance 

Cov[g0{a{w,u,-)),gg{a{w,u',-))) = 9{E^d-{w,u,v)a{w,u',v)). (1.37) 

Let us consider independent standard Gaussian random variables z and z' and define 

G^,{a{w, u, ■)) = g^'iaiw, u, ■)) + ^(^'(1) - Ci^vCriw, u, vf)) '^' (1.38) 

and 

Ge{a{w, u, ■)) = ge{Hw, u, ■)) + ^(^(1) - e{E,a{w, u, vf))'^^ (1.39) 

For simplicity of notations we will keep the dependence of G^i and Gg on z or z' implicit. Let 
G^'^i and Gg^i be independent copies of these processes. Random variables z and z' will play 
the role of replica variables similarly to u and for this reason in the Sherrington-Kirkpatrick 
model we will denote by E' the expectation in u, z and z'. The main purpose of introducing 
the second term in (1.38) and (1.39) is to match the variances of these Gaussian processes, 
^'(1) and ^(1), to variances in (1.34) for cr^ = cr^. 

As in the setting of diluted models, consider arbitrary n,m,q,r > 1 such that n < m. 
For each / < q consider an arbitrary subset C/ C {1, . . . ,m} and let C/ and Gf be defined 
as in (1.20). Let ai = a{w,u,Vi). For / < q define 

Ui = E'W thG^,^i{a{w,u,-)) J] a^En^r (1-40) 

i&C} i€Cf 

where 

£n,r = exp(^\ogchG^>^i{a{w,u, ■)) + ^ G'e,fc(o-(w,M, ■))) (1-41) 

i<n k<r 

and let V = E'Sn,r- If -M denotes the set of possible limits of fiN corresponding to the 
Hamiltonian (1.31) perturbed by (1.35) then the following holds. 



Theorem 3 For any /i G A^ and a = a^ we have 



Rk, Ui 



^\[^'\{<^^=^-^- (1-42) 

l<q ieCi 

Let A4inv be the family of distributions defined by the invariance properties (1.42), so that 
Theorem 3 proves that Ai C J\Ainv H we define 

V{fi) = log2 + ElogE'chG'^/(CT^(w,M, ■))-ElogE'expGe(^M(^'«'-)) (1-43) 

then we have the following representation for the free energy in the Sherrington-Kirkpatrick 
model. 
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Theorem 4 We have, 



lim Fjv = inf P(^) = inf P(/i). (1.44) 



As in the case of diluted models above, let us describe several special cases of (1.42). If r = 
and sets Ci are such that C^ C {n + 1, . . . , m} for all / < g then (1.42) becomes 

If we set n = then (1.42) becomes 

nT-r TT;^ E'rr ,-^ (J.; exp V,^ Gfl i.(o"(w, M, ■)) 

,<, ,i^, (E'expE.<.G,,.(a(t.,«,-)))^ ^ ' 

Again, these equations can be interpreted as the invariance of the spin distributions under 
various random changes of density. Finally, if we set r = and n = m then (1.42) becomes 

,<, Ir^" (E'n.<„chG,,,(a(«;,.,.)))^ " ^''''^ 

The meaning of this self-consistency equation is that the joint distribution of spins generated 
by a function ^(w, -u, v) coincides with the distribution of spins generated by thG^/(a'(w, -u, ■)) 
under a properly interpreted random change of density and we will discuss this interpretation 
in more detail below under the Parisi ultrametricity hypothesis. The choice of parameters in 
equation (1.47), most importantly n = m, will be the key to the following special property 
of the Sherrington-Kirkpatrick model. 

Theorem 5 For any fi G Aiinv, the joint distribution of {Rf°i/)i^i'>i defined in (1.8) for 
a = o-fj, uniquely determines fj, and thus the joint distribution of all multi- overlaps. 

The fact that the joint distribution of overlaps determines fi leads to a natural addition to 
the statement of Theorem 4. It will be clear early in the proof of Theorem 4 that V{fi) for 
fj, E Ai depends only on the distribution of the array (1.8) for a = a^ and, as a result, 
one can express the free energy in (1.44) as the infimum over a family of measures Ai'inv 
defined completely in terms of the invariance of the joint overlap distribution and such that 
■M-inv ^ -^'inv -^o^ this purpose one does not need the self-consistency part of the equations 
(1.42), so we will only use the case when Cf = Ci in (1.20) for all /. Let us consider processes 
G^i and Go in (1.38), (1.39) defined in terms of replicas (ui), (zi) and [z'l) of u, z and z' ^ 
namely, 

G^,{cy{w, ui, ■)) = g^'{cT{w, uu ■)) + z, (e'(l) - e'(E.a(w, m, vf)) "^ (1.48) 

and 

Ge{(y{w, uu ■)) = 9e{<y{w, uu ■)) + 4 {e{l) - e{W.,<y{w, m, vf)) ^'\ (1.49) 

Let F = -F((-Rn/)«,F<(/) be an arbitrary continuous function of the overlaps on q replicas. Let 
f/ = E'F J] exp (J2 log ch G^,,{cxiw, ui, ■)) + J2 ^eA^i^^ ^h ■))) • (1-50) 

K(j i<n k<r 
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Then the condition 

EF = E([//\/«) (1.51) 

for all q,n,r and all continuous bounded functions F defines the family Ai'inv Equation 
(1.51) is obviously implied by (1.42) which contains the case of polynomial F simply by 
making sure that Cf = Ci, so M.inv ^ ■^'inv Then, one can add 

lim Fn = inf P(/x) (1.52) 

to the statement of Theorem 4. This together with Theorem 5 shows that in the Sherrington- 
Kirkpatrick model the role of the order parameter is played by the joint distribution of 
overlaps rather than the joint distribution of all multi-overlaps or the generic functional 
order parameter a^. This gives an idea about how close this point of view takes us to the 
Parisi ansatz [23] where the order parameter is the distribution of one overlap. Since we 
can always ensure that the Ghirlanda-Guerra identities [12] hold by adding a mixed p-spin 
perturbation term (see (1.53) below), the remaining gap is the ultrametricity of the overlaps, 
since it is well-known that the Ghirlanda-Guerra identities and ultrametricity determine the 
joint distribution of overlaps from the distribution of one overlap (see, for example, [5] or 
[7]). If one can generalize the results in [18] and [27] to show that the Ghirlanda-Guerra 
identities always imply ultrametricity, (1.44) would coincide with the Parisi formula proved 
in [25]. 

The Ghirlanda-Guerra identities and stochastic stability. Let us mention that 
the Ghirlanda-Guerra identities and stochastic stability can also be expressed in terms of 
the generic functional order parameter a. We will use a version of both properties in the 
formulation proved in [27]. Let us now consider a different perturbation term 

HU<^) = SnJ2 f^N,pK,p{'^) (1.53) 

p>i 

where 

-Hn,p{(t) = ^(p_i)/2 Yl 9i...,^,(r,,...ai^, (1.54) 

l<ii,...,ip<N 

are independent copies of (1.32). When 6^ -^ this perturbation term is of smaller order 
than (1.31) and does not affect the limit of the free energy. However, the arguments in the 
proof of the Ghirlanda-Guerra identities and stochastic stability in [27] require that 6n does 
not go to zero too fast, for example the choice of 5n = N~^^^^ works. Then, Theorem 2.5 in 
[27] states that one can choose a sequence [3^ = {.Pn.p) such that \I3n,p\ < 2"^ for all A^ and 
such that the following properties hold. First of all, if (■) is the Gibbs average corresponding 
to the sum 

-H'^{ct) = -Hm{(t) - H%{(t) (1.55) 

of the Hamiltonians (1.31) and (1.53) and F is a continuous function of finitely many multi- 
overlaps (1.6) on replicas cr^, . . . , cr" then the Ghirlanda-Guerra identities 

°° 1=2 
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hold for all p > 1. Now, for p > 1, let Gp{(T) be a Gaussian process on S^r with covariance 

¥.Gp{(T^)Gp{(T^) = Rl^ (1.57) 

and for t > let (■)t denote the Gibbs average corresponding to the Hamiltonian 

-K,ia) = -H'^icT)-tG,icT). 
Then, in addition to (1.56), the following stochastic stability property holds for any t > 0, 

lim \E{F)t-E{F)\ = 0. (1.58) 

Let fi]\f be the joint distribution of spins (1.2) corresponding to the Hamiltonian H'^{a) and 
M. be the set of all limits of {^n)- Then both (1.56) and (1.58) can be expressed in the limit 
in terms oi a = a^ for any /i G A^ as follows. First of all, (1.56) becomes the exact equality 
in the limit by comment above (1.7), 

1 1 " 

EF(i??"„ , i)P = - EF MRf'.f + - V EF(i?~)^ (1.59) 

' n ' n ^-^ 

1=2 

Stochastic stability (1.58) can be expressed as follows. For w G [0, 1], let gp{a{w,u, ■)) be a 
Gaussian process indexed by -u G [0, 1] with covariance 

Cov{gp{a{w, u, ■)),gp{a{w, u', ■))) = {Eya{w, u, v)a{w, u', v)Y (1.60) 

and, as in (1.38), let 

Gp{a{w, u, ■)) = gp{a{w, u, ■)) + z{l- {E,a{w, u, vff) ^'\ (1.61) 

Then (1.58) implies the following analogue of Theorem 3. 
Theorem 6 For any ^ E Ai and a = a^ we have for all p > 1 and t > 0, 

E'Y\ai = E ^-'' , ^^^^^' \ — ^\ , '' . (1.62) 

,<^ Al (E'exptG,(c7(^, «,-)))« ^ ' 

Note that we could also add in (1.55) the previous perturbation term (1.35), which would 
simultaneously ensure that the invariance (1.42) and representation (1.44) hold. The proof 
that (1.58) implies (1.62) will not be detailed since it follows exactly the same argument as 
the proof of Theorem 3 (we will point this out at the appropriate step in Section 3.4). Note 
that (1.62) is more general than (1.46), which shows that the invariance of Theorem 3 is 
related to the stochastic stability (1.58). It is interesting to note, however, that the size of 
the perturbation (1.35) that ensured the invariance in (1.42) was of arbitrarily smaller order 
than the original Hamiltonian (1.31) since cat could grow arbitrarily slowly while perturbation 
(1.53) must be large enough since 5n can not go to zero too fast. Moreover, the form of the 
perturbation (1.35) plays a crucial role in the proof of the self-consistency part (1.47) of 
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the equations (1.42) which will allow us to give an explicit construction of the functional 
order parameter a below under the Parisi ultrametricity hypothesis. The special case of the 
stochastic stability (1.62) for the overlaps (rather than multi-overlaps as in (1.62)) was the 
starting point of the main result in [3] under certain additional assumptions on a. 

Let us make one more comment about the Ghirlanda-Guerra identities (1.59) from the 
point of view of the generic functional order parameter a. Equation (1.56) always arises as 
a simple consequence of the following concentration statement either for the perturbation 
Hamiltonian (1.54) (see [27]), 



lim E 



m 



N,p 



N 



E 



m 



N,p 



N 



0, 



or for the Hamiltonian in (1.32), 



lim E 



H 



N,p 



N 



-E 



H 



N,p 



N 



0, 



;i.63) 



;i.64) 



which was proved in [22] for any p such that f3p ^ in (1.31) (the case oi p = 1 was first 
proved in [8]). One can similarly encode the limiting Ghirlanda-Guerra identities (1.59) as a 
concentration statement for the Gaussian process Gp{(j{w,u^ ■)) in (1.61) as follows. 

Theorem 7 Assuming (1.62), the following are equivalent: 

(1) the Ghirlanda-Guerra identities (1.59) hold; 

(2) for all p> 1, 



^ Gp{a{w, u, ■)Y exp tGp{a{w, u, ■)) _ /Gp{a{w,u,-))exptGp{a{w,u,-))y 



W exptGp{a{w,u,-)) V E' exptGp{a{w,u,-)) 

is uniformly bounded for all t > 0, in which case it is equal to 1. 



;i.65) 



The result will follow from a simple application of the Gaussian integration by parts and the 
main reason behind this equivalence will be very similar to the proof of the Ghirlanda-Guerra 
identities for Poisson-Dirichlet cascades in [28]. 

Spin distribution under ultrametricity. Finally, let us go back to the self-consistency 
equations (1.47) and show that they can be used to give an explicit formula for the function 
a, or the distribution of spins, under the Parisi ultrametricity hypothesis and the Ghirlanda- 
Guerra identities. We assume that the reader is familiar with the Ruelle probability cascades, 
so we will omit all details and simply refer to previous work. Equation (1.8) defines some 
realization of the directing measure of the overlap array in the following sense. If we think 
of a^w, u, ■) as a function in H = >C^([0, 1], dv) then the image of the Lebesgue measure on 
[0, 1] by the map u — )■ a{w,u, ■) defines a random probability measure r/^ on H. Equation 
(1.8) states that the overlaps can be generated by scalar products in H of an i.i.d. sequence 
from this random measure. Any such measure r]y^ defined on an arbitrary Hilbert space is 
called a directing measure of the overlap array {Rfli). It is defined uniquely up to a random 
isometry (see, for example. Lemma 4 in [21], or in the case of discrete overlap the end of the 
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proof of Theorem 4 in [18]). By Theorem 2 in [18], the Ghirlanda-Guerra identities imply 
that 

'Eya{w,u,vY = q* a.s. (1.66) 

where q* is the largest point in the support of the distribution of R'^2 ^''^d, therefore, equation 
(1.47) can be slightly simplified, 

eTTe'TT- - E ^'^g ^' ^^^^' th^g.,,(a(w, u, ■)) lli<n ch ^e.»(^(^^^ u, ■)) 
l\ kk""^ (E'n.<„ch^^,,(a(^, «,-)))« • ^- ^ 

The key observation now is that the right hand side of (1.67) does not depend on the 
particular realization of the directing measure since the Gaussian process g^' is defined by its 
covariance function (1.36) which depends only on the Hilbertian structure of the directing 
measure. Let us first interpret the right hand side of (1.67) when the overlap distribution is 
discrete, 

^(^^2 = ^0 =mi+^-mi (1.68) 

for some < gi < g2 < • • • < ^fc ^ 1 and = mi < . . . < rrik < iTLk+i = 1. In this case it is 
well-known that the directing measure is given by the Ruelle probability cascades, of course, 
assuming the Ghirlanda-Guerra identities and ultrametricity (see, for example, [3], [18], [25] 
or [28]) and, therefore, (gs^'^i) are the usual Gaussian fields associated with the cascades. 
The Ruelle probability cascades is a discrete random measure with Poisson-Dirichlet weights 
(wa) customarily indexed by a G N^, where k is the number of atoms in (1.68), so that the 
Gaussian fields are also indexed by a, {g^/i{a)) and {gg^i{a)). By definition of a directing 
measure 7]^,, the expectation E' in u plays the role of averaging with respect to these weights, 
so that the right hand side of (1.67) can be rewritten as 

^ riKg Eg Wg U^eC^ ^^ 9(',^i^) Ui<n ^^ d^'A^^) ^ gg. 

(Ea^«ni<„ch^5',i(a)))« 

This in its turn can be rewritten using well-known properties of the Ruelle probability 
cascades, in particular. Lemma 1.2 in [17] which is a recursive application of Proposition A. 2 
in [6] . If we denote 

w = = 

then the point processes 

have the same distribution, where {gii j(q;)) is a random field (no longer Gaussian) associated 
with the Ruelle probability cascades defined from the Gaussian field {g^>^i{a)) by an explicit 
change of density (see equation (7) in [17]). Therefore, (1.69) is equal to 

Em2w^l[thg'^,/a) (1.70) 

l<q a i&Ci 
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which can now be interpreted as an exphcit construction of a{w,u,v). First coordinate w 
corresponds to generating the Ruelle probabihty cascade r]^, second coordinate u plays the 
role of sampling a point in this measure, or its index a, according to weights (wa), and the 
last coordinate v corresponds to generating a random field gi, associated with the cascades. In 
particular, (1.70) implies that the limiting distribution of the Gibbs averages ((Tj) of finitely 
many spins 1 <i <n coincides with the distribution of 

^w„th^^, .(a) for 1 < i < n. (1.71) 

a 

This can be thought of as the generalization of the high temperature result. Theorem 2.4.12 
in [24], under the assumption of the Parisi ultrametricity. It will be clear from the proof of 
Theorem 3 that the right hand side of (1.67) is continuous with respect to the distribution of 
the overlap array (1.8) and, on the other hand, the ultrametricity allows one to approximate 
any overlap array by a discretized overlap array satisfying (1.68) uniformly while preserving 
ultrametricity and the Ghirlanda-Guerra identities. Therefore, one can think of the case of 
an arbitrary distribution of the overlap simply as a limiting case of the above construction 
for discrete overlaps. 

D 

While many technical details will be quite different, the main line of the arguments in the 
setting of the Sherrington-Kirkpatrick model in Section 3 will be parallel to the arguments in 
Section 2 for diluted models. A reader only interested in the Sherrington-Kirkpatrick model 
should read Lemma 2 before skipping to Section 3. 

Acknowledgement. The author would like to thank Tim Austin for motivating this 
work and Michel Talagrand for asking good questions and making many suggestions that 
helped improve the paper. 



2 Diluted models. 

2.1 Properties of convergence. 

Let us first record a simple consequence of the fact that the distribution of the array in (1.3) 
is the limit of the distribution of spins (o"') under the annealed product Gibbs' measure. As 
usual, (■) will denote the expectation with respect to the random Gibbs measure. 

Lemma 1 Let hi, ... , hm '■ { — 1, +1}" — > [~-^) ^] ^c some bounded functions ofn spins and 
let h be a continuous function on [—K,K]"^. Let cr = (o'j)i<j<„ and let s = (Sj^)i<j<n defined 
in (1.3) for some fi E Ai. If fJ^N converges to fi over subsequence (Nk) then 

lim Eh{{hiia)),...,{h^{a)))=Eh{E'hi{s),...,E'h^{s)). (2.1) 

Proof. Since it is enough to prove this for polynomials h and since each hi is a polynomial 
in its coordinates, this statement is simply a convergence of moments 



v"" Kn-.' ='^n»' 
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where the product is over a finite subset of indices (i,/). 

D 

We will often use this lemma for random functions h, (hi) independent of all other 
randomness, simply by applying (2.1) conditionally on the randomness of these functions. 
Justifications of convergence will always be omitted because of their triviality. 

Another simple property of convergence of spin distributions under the annealed Gibbs 
measure in diluted models is that adding or removing a finite number of terms to the Poisson 
number of terms 7r(a;A^) or 7r(cjv) in (1.15) does not affect the limit of these distribution over 
any subsequence for which the limit exists. Let {Nk)k>i be any such subsequence and let 
n, m be fixed integers. In fact, it will be clear from the proof that one can let n, m grow with 
iVfc, but we will not need this. Let H'^ be defined exactly as (1.15) only with Ti{aN) + n 
terms instead of Ti{aN) in the first sum and ti{cn) + rn instead of ti{cn) in the perturbation 
term and let (■)' denotes the corresponding Gibbs' measure. 

Lemma 2 For any hounded function h of finitely many spins in array (cr') we have 

lim |E(/i)'-E(/i)| =0. (2.2) 

Proof. For certainty, let us assume that n, m > and \h\ < 1. If we denote by (■)jj the 
Gibbs average conditionally on 7r(Q;A^) = i and tx{cn) = j then 

E{h) = ^7r(aiV,2)7r(c^,j)E(/i)y 

where from now on 7r(A, k) = \^e~^ /k\ and 

^(^)' = y^ Ti{aN ,i)Ti{cN , 3y^{h)i+n,j+m 

= y^ TT{aN,i — n)n{cN,j — rn)E,{h)ij. 

i>n,j>m 

Therefore, 

\E{hy-E{h)\ < ^7r(aA^,i) + ^7r(c7v,j) 

i<n j<m 

+ y^ {T^ioiN^i — n)-K{cN,j — m) — 7r{aN,i)n{cN,j)\ 

i>nj>m 

< ^7r(aAr,2) + ^7r(c^,j) 

i<n j<m 

+ ^\7r{aN,i-n) -7r{aN,i)\ + ^ \7r{cN,j -m) -7v{cn,J)\- 

i>n j^rn 

The first two sums obviously go to zero. One can see that the third sum goes to zero as 
follows. Poisson distribution with mean aN is concentrated inside the range 



aN - ^/NlogN <i<aN+ ^/NlogN. (2.3) 
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If we write 



\7i(aN,i-n)-n(aN,i)\ =7i(aN,i) 1 - -^^— (aA^)"" (2.4) 

[I — ny. 



then it remains to note that 



'■ (aiV)-"=^(^-^V-(r^^'^^l 



{i-n)r ' {aNy 

uniformly inside the range (2.3). Similarly, the last sum goes to zero which finishes the proof. 

Remark. Lemma 2 implies that (2.2) holds even if n is a random variable. We will use 
this observation in the case when iJ^ is defined exactly as (1.15) only with 7r(aiV + n) terms 
instead of 7r(Q;iV). In fact, in this case one can write 

^(^)' = 5Z 7r(aiV + n,i)7r(cjv,j)E(/i)ij+m 

and instead of (2.4) use 

|7r(Q;A^ + n, i) — 7r(«A^, i) \ = n{aN, i) 






aN 



and notice that again the last factor goes to zero uniformly over the range (2.3). Similarly, 
one can have 7r(cAr + n) instead of tt{cn) terms in the perturbation Hamiltonian without 
affecting convergence. 

D 

Due to the perturbation term (1.14) the following important property of convergence 
holds. 

Lemma 3 // fiN converges to fi over subsequence (Nk) then it also converges to fi over 
subsequence {Nk + n) for any n> 1. 

Proof. We will show that the joint moments of spins converge to the same limit over sub- 
sequences that differ by a finite shift n. Let h = YljKq^j where hj = Yli^c '^l '-'^^^ some 
finite sets of spin coordinates Cj. Let us denote by {•)n the Gibbs average with respect to 
the Hamiltonian (1.15) defined on A^ coordinates. We will show that 

lim mh)N+n-Mh)N\ = 0. 

Let us rewrite E(/i)Ar_|_„ by treating the last n coordinates as cavity coordinates. Let us 
separate the 7r(Q;(A^ + n)) terms in the first sum 

E ^'^•K.'---'^vJ (2-5) 

k<-K(a(N+n)) 

of the Hamiltonian HN+n{(^) in (1-15) into several groups: 
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(1) terms for k such that all indices ii^fc, • • • , ip,k < N-, 
For 1 < / < n : 

(2/) terms with exactly one of indices ii^k) ■ ■ ■ , ip,k equal to A^ + / and all others < A^; 

(3) terms with at least two of indices ii^k-, ■ ■ ■ , ip,k > N. 
The probabilities that a term is of these three type are 



Pi = T^ ) , P2,l =P^r^ T^ ) > P3 = l-Pl - yp2,l- 

\i\+n/ i\+n\i\+n/ ^-^ 



Kn 



Therefore, the number of terms in these groups are independent Poisson random variables 
with means 

a{N + n)pi = a{N + n-np)+0{N~^), a{N + n)p2,i = ap + 0{N-^), a{N + n)p3 = 0{N-^). 

We can redefine the number of terms in each group to be exactly of means a{N + n — np), ap 
and since asymptotically it does not affect E(/i)7v+n- Thus, if we write a = (p, e) G SAr+„ 
for the first A^ coordinates p = (pi, . . . , p^) and the last n cavity coordinates £ = (ei, . . . , e„) 
then (2.5) can be replaced with 

Yl 0k{ph,„...,pi^J + Yl Yl ^k,l{^l^ Ph,k,n ■ ■ ■ , Pip-i,kj) (2-6) 

k<w{a{N+n—np)) l<n k<iTi{ap) 

where indices ii^k, ■ ■ ■ , ip,k and ii^k,h • • • ? ip-i,k,i are all uniformly distributed on {1, . . . , A^}. 
Let us now consider the perturbation term in (1.15) 

J2 logAv.exp J2 ^M(^>o^ji,M'---'^ip-i,fc.^) (2-7) 

«<7r(cjv+„) k<Tti{ap) 

where ji^k,i, ■ ■ ■ ,jp~i,k,i are uniformly distributed on {1, . . . , A^ + n}. Here, we used indepen- 
dent copies VT; and 6k,i since ni and 6k,i were already used in (2.6). The expected number of 
these indices that belong to {A^ + 1, . . . , A^ + n} is C]\f+n<^p{p — l)n/N — )■ which means that 
with high probability all indices belong to {1, ... , A^}. As a result, asymptotically K{h)]\r^n 
will not be affected if we replace the perturbation term (2.7) with 

J2 logAv.exp J2 ^k,i{^^Pji,k,v-^Pjp-i,k,i) (2-8) 

l<TT{cN+n) k<wi{ap) 

where ji,k,i, ■ ■ ■ ,jp-i,k,i are uniformly distributed on {1, ... , A^}. Thus, we can assume from 
now on that 'K{h)N+n is computed with respect to the Hamiltonian which is the sum of (2.6) 
and (2.8). If {■)'j^ denotes the Gibbs average on Sat with respect to the Hamiltonian 

-Hn{p) = Yl ^fe(Pn,.'---'PvJ 

k<TT{a{N +n—np)) 

+ Y logAv.exp Y ^m(^>Pji.m'---'P^p-i.m) 

Z<7r{cjv+„) k<-ki{ap) 
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then each factor in 



{h)N+n = ll{h,)N+n = IK H ''^) N+n = IK H P^) N+n 

j<q j<q ieCj j<q i£CU 



can be written as 



^lljGCj Pi-^^e exp 2^;<„ Z^k<ni{ap) "kA^li Pil,k,li ■ ■ ■ i Pip-l,k,l, i -p-j- „ 

{hj)N+n = -, TT ^ (11 PyN 



Av, exp X;i<„ Efc<7r,(Qp) ^k,ii^h Pii,fe,p • • • , Pip-i,k,i)/^ »e^^- 

where (■)^ is the Gibbs average on Sat corresponding to the Hamiltonian 

-^Af(p) = -^iv(p) + Xl^°sAveexp Y^ ^fc,K^,Pii.M'---'PVi,M)- 

l<n k<TTi{ap) 

But this Hamihonian differs from the original Hamihonian (1.15) only in that the first sum 
has 7T{a{N + n — np)) terms instead of 7r(Q;A^) and the perturbation term has 7r(cjv+n) + n 
terms instead of 7r(cjv). Therefore, appealing to Lemma 2 and remark after it shows that 
E(/i)^ is asymptotically equivalent to E(/i)jv and this finishes the proof. 
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2.2 Lower bound. 

Lemma 4 There exists ^ E M. such that lim^v-^oo F^ > 'P{^). 

Proof. We will obtain the lower bound using the well-known fact that 

lim Fn > liminf((A^+ l)FAr+i - NFn) = liminf Elog ^^^. (2.9) 

Suppose that this lower limit is achieved over subsequence {Nk) and let /i G A^ be a limit 
of (/iAr) over some subsubsequence of {Nk). Let a = a^. The considerations will be very 
similar to the proof of Lemma 3. Let us consider E log Zjy+i and let us start by separating 
the 7r(a(iV-|- 1)) terms in the first sum in the Hamiltonian H^j^i in (1.15) into three groups: 
(1) terms for k such that all indices ii^k-, ■ ■ ■ ■,ip,k < N\ (2) terms with exactly one of indices 
^i,fc, . . . , ip,k equal to A^ + 1; (3) terms with at least two of indices i\^k, ■ ■ ■ , ip,k equal to A^ + 1. 
The probabilities that a term is of these three types are 

N \p I r N \p-i 



correspondingly. Therefore, the number of terms in these three groups are independent Pois- 
son random variables with means 

a{N + l)pi = a{N-p + l) + 0{N~^), a{N + l)p2 = ap + 0{N-'), a{N + l)p3 = 0{N-^). 
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For simplicity of notations, let us pretend that the number of terms in each group is exactly of 
means a{N—p), ap and since it will be clear from considerations below that asymptotically 
it does not affect the limit in (2.9). If we write cr = {p,e) G S^v+i for p G Sat and e G 
{—1, +1} then we can write the first term in Hn+i{(t) as 

Yl ^fc(Pn.fe'---'PvJ+ Yl ^^(^'Pii,fe'---'Pip-i.J (2-10) 

k<n{a{N-p+l)) k<n{ap) 

where indices ii^k, ■ ■ ■ , ip,k and ji j^, . . . ,jp-i^k are uniformly distributed on {1, . . . , A^}. Simi- 
larly, we could split the 7r(c7v+i) terms in the perturbation Hamiltonian (1.14) into indices / 
for which all ii,k,i, ■ ■ ■ , ip-i,k,i < N and indices / for which at least one of these indices equals 
N + 1. However, as in the proof of Lemma 3, since with high probability all these indices 
will be < iV and |cAr+i — CAr| — )■ 0, we can simply replace the perturbation term with 

Y logAveCxp Y ^kA^^Ph,k.n---,Pip-i,k,i) (2-11) 

where ii^k,i, ■ ■ ■ , ip-i,k,i are uniformly distributed on {1, ... , A^}. Let (■)' be the Gibbs average 
on Ejv corresponding to the Hamiltonian 

-H'n{p) = Yl ^k{pi,^„ . ■ . , Pi^^j 

k<Tr{a{N-p+l)) 

+ Y logAv^exp Y ^kA^^Ph,k.n---,Pip-i,k,i) 
i<7r(c]v) k<ni{ap) 

and Z'^ be the corresponding partition function. Then, 

Elog^ = Elog(5^exp Y ^k{e,p,,,,,...,p,^_,_j)'. (2.12) 

^ e=±l k<7r(ap) 

Conditionally on ^{ap) and {6k) and on the event that all indices ji^k, ■ ■ ■ ,jp-i,k are different. 
Lemmas 1 and 2 imply that (2.12) converges to 

ElogE' Y 6^P Y ^'^•(^' '^1''^' ■ • ■ ' -^p-i.fc)- 

£=±1 k<n{ap) 

For large A^, indices ji^k, ■ ■ ■ ,jp-i,k will all be different for all k < 7f{ap) with high prob- 
ability and, therefore, this convergence holds unconditionally. Similarly, one can analyze 
E,log{Z i\f/Z'j^). Let us split the first sum in the definition of —Hj^{p) in (1.15) into two sums 

Y ^fe(Pn.fe'---'Pip.J + Y ^kiPn^k,---,Pjp^J 

k<TT{a{N-p+l)) k<TT{a{p-l)) 

where indices ii^k, ■ ■ ■ , ip,k and ji^fc, • • • ,3p,k are uniformly distributed on {1, . . . , A^}. There- 
fore, 

Elog|^ = Elog(exp Y ^>^iPn,u.---.P,,^S)'- (2-13) 

^ k<7T{a{p-l)) 
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Again Lemmas 1 and 2 imply that this converges to 



ElogE'exp 2_^ Ok{si,k,---,Sp^k) 

k<TT{a{p— 

and this finishes the proof of the lower bound. 

D 

If we knew that /i G A^ is the unique limit of the sequence (/iA?), this would finish 
the proof of the first half of Theorem 2, since limTv-i-oo -^Af = liniAr_i.ooElogZ7v+i/^Af when 
the limit on the right exists. However, the proof of the general case and the second half of 
Theorem 2 will require more work. Before we move to the upper bound, let us record one 
more consequence of the argument in Lemma 4. For n > 1, let us define 

P„(/i) = log2 + -ElogE'Avsexp VAi(£i) -- ElogE'exp V^^. (2.14) 

n ^-^ n ^-^ 

i<n i<n 

The following holds. 

Lemma 5 For all ii G A^, VnifJ^) = Vi^jj) for all n > 1. 

Proof. We will only give a brief sketch since this will be proved for all ^ (z Aimv in Lemma 7 
below. What we showed in the proof of Lemma 4 is that if ^n converges to /i over subsequence 
(iVfc) then ElogZAT+i/ZAT converges to V^jj) over the same subsequence. Similarly, one can 
show that, given n > 1, over the same subsequence 

-(ElogZ^+„ - ElogZ^) ^ P„(/x). 
n 

The only difference is that we split the terms in the Hamiltonian HN+n{(^) into groups as in 
Lemma 3, i.e. instead of group (2) we will have n groups each consisting of the terms with 
exactly one of the indices ii^k, • • • , ip,k equal to A^ + / for / = 1, . . . , n. On the other hand, if 
we write 

1 1 " 

-(E log Zn+u - E log Zn) = - y^(E log Zn+i - E log ^at+z-i) 
n n ^ — ^ 

1=1 

then repeating the proof of Lemma 4 one can show that for each term on the right hand side 

hm Elog^^^^^ = P(/i), 

where instead of /iTv^ — ^ At one has to use that fJ'Nk+i-i "^ A* which holds by Lemma 3. This 
finishes the proof. 
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2.3 Upper bound and free energy. 

Since the perturbation term in (1.15) does not affect the hmit of free energy, we will now 
ignore it and consider free energy F^ defined for the original unperturbed Hamiltonian (1.9). 
Recall Ai{e) and Bi defined in (1.18) and (1.19). 

Lemma 6 For any function a : [0, 1]^ — )■ {—1, +1} we have 

Fjv<log2 + ^ElogE'Av,exp^Ai(£,)--Elogrexp^5,. (2.15) 

i<N i<N 

Remark. In general, this upper bound does not decouple and depends on A^ since all Si^k,i 
and Si^k,i defined in (1.16) and (1.17) depend on the same variable u in the second coordinate. 
We will see that the proof of the upper bound (2.15) does not to work if one tries to replace 
u by independent copies Ui in the definition of Ai{e) and Bi. For a = o"^ for /i G A^, Lemma 
5 implies that this upper bound does not depend on A^ and, thus, Fjv < 'P{^). Together with 
the lower bound of Lemma 4 this proves that 

lim F/v = inf Vi^jj). 

To prove the second part of Theorem 2, we will show in Lemma 7 below that the invariance 
properties in (1.23) imply that Vn{f^) = 'P{^) for /i G Aiinv as well which will finish the 
proof of Theorem 2. 

D 

Proof of Lemma 6. A proof by interpolation is a slight modification of the proof in 
[19]. For t G [0, 1], let us define similarly to (1.18) and (1.19) 

"^i(^) = X] ^fc,i(^, ■5j,fc,i,...,Si,fc,p-i) (2.16) 

k<-K,({l-t)pa) 

and 

Bi= 2_^ Ok,i{si^k,i-,- ■ ■ -iSi^k^p)- (2.17) 

k<-Ki{t{p-l)a) 

Consider an interpolating Hamiltonian 

-if^,,(^)= Y. OM,^,,...,a,^^,) + Y,A\{ai) + Y,B\ (2.18) 

k<iT{taN) i<N i<N 



and let 



Since, clearly. 



ifit) = ^ElogE' J2 eM-HN,M)- 



N' 



if{l) = FN + -E log E' exp J2 Bi 



N 

i<N 
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and 



ip{0) = log 2 + -ElogE'Av.exp^ AN 

i<N 



it remains to prove that v^'(t) < 0. Let us consider the partition function 

Z = ^ exp{-HN,t{cr)) 



and define 



7 = Zl 7^ = 7\ and 7^ = 7\ 

™ \n{taN)=m' hk ^ \'„^{(l-t)pa)=m ^^ *.fc l7r,(t(p-l)Q)=m' 



If we denote the Poisson p.f. as 7r(A, A;) = {\^ /k\)e ^ then 

E log Z = ^ Ti{taN, m)E log E'Z„ 

m>0 

and, for any i < N, 



ElogZ = ^vr((l -t)pa,m)ElogE'Z^ 
and 



A 
m>0 



ElogZ = J2 ^(tiP - l)a,m)ElogE'Zf„. 
Therefore, we can write 

m>0 i<N m>0 

j<Af m>0 



m>0 



^(7r(toA^,m- l)/(m > 1) - 7r(toiV,m))ElogE'Z„ 

m>0 

j<Af m>0 

+ (p - l)a^ 5^ Yl (^(^(P - 1)"' ^ - 1)^(^ ^ 1) - ^(^(P - 1)"' ^))ElogE'Zj„ 

j<Af m>0 

a ^ 7r(toiV, m)Elog(E'Z„,+i/E'Z„) 

m>0 

-^"^ E E ^((1 - i)P«,^)Elog(E'z4+i/rZ^) 

j<Af m>0 

+ (p- l)a^5^5^7r(t(p- l)a,m)Elog(E'Zf„+i/E'Zfj 

j<Af m>0 

aE log -—^ - pa— > E log -^^^ + ip - 1 «E log -—^ 2. 19 



E'Z A^ ^^ E'Z ' ' E'Z 
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where ^+i, -^j^+i and Z^-^^ contain one extra term in the Hamiltonian in the corresponding 
Poisson sum. Namely, 

Z+i= ^ exp6'(o-ij,...,ajJexp(-i7Ar,i(cr)), 

^+1 = ^ exp 9{si, ...,Sp) exp{-HN,t{o-)), 

where random function 9 and indices ii, . . . ,ip uniform on {1, ... , A^} are independent of the 
randomness of the Hamiltonian Hf^ff If, for a function f oi (t,u and (x), we denote by {f)t 
the Gibbs average 

then (2.19) can be rewritten as 

a E log(exp e{ai^ , . . . , ciij)^ - pa— J^ E log(exp 9{ai, si, . . . , Sp_i) )^ 

+ {p- l)aE log(exp e{si, ..., Sp))^. (2.20) 

By assumptions (1.10) and (1.11) we can write 

log(exp^(ai,,...,aiJ)^ = loga + log(l + 6(/i(CTiJ.../p((TiJ)J 

= loga-^^^(/i((TiJ.../p((Tij)". 
n>l 

Using replicas a^ui and (x'), we can write 

(A(a,) . . . /,(a,J>; = (H /i«) . . . /p«)>, 

l<n 

and, thus, 

i^ 5^ (/i(-j.../p(-.j>:=(n^.-4 

ii,...,ip<N j<p 



where 



A,,„=A,,„(^\...,^-)=i5^n/^(^')- 

i<N Kn 



Denote by Eq the expectation in /i, . . . , fp. Since fi, ■ ■ ■ , fp are i.i.d. and independent of the 
randomness in (■)(, 

i<p i<p 
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where Bn = KoAj^n- Therefore, since we also assumed that b is independent of /i, . . . , /p, 



1 ( U\n 

Eo^ E log(exp^K,---,^v)X = Eologa-J]^(i?^X. (2. 

il,...,ip<N n>l 

A similar analysis applies to the second term in (2.20), 

log(exp^(cTi,Si,...,Sp_i))^ = loga-^ (fp{(Ti) JJ fj{s 



n \ -^-^ / 1 

n>l iS:P— 1 



n>l l<n l<n j<p—l 



where in the last equality we again used replicas cr^ui and (a;'), for example, compared to 
(1.16), s^- is now defined by s^- = (7{w,ui,Vj,x'-j). Thus, 



1 V ( — b)"' I -I— r -I— r \ 

{Note: It was crucial here that s^- do not depend on i through independent copies Ui rather 
than the same u. It is tempting to define the interpolation (2.18) by using independent Ui 
for i < N since this would make the upper bound in (2.15) decouple but the proof would 
break down at this step.) In addition to /i, . . . fp, let Eq also denote the expectation in {vj) 
and (a;^) in s^ but not in sequences {v), (x) in the randomness of (■)(. Then, 

Eo^ E l°g(^^P ^(^*' ^1' • • • ' ^P-i)>t = Eo loga - E ^^\^«(^")''" /, (2.22) 

where 

Cn = CniyO, Ml, ... , M„) = Eq J]^ f j{s\) = Eq J]^ fj{cr{ui, Ul, Vj, X^-)) 

obviously does not depend on j. Finally, in absolutely similar manner 

Eolog(exp^(si, . . . , sp))^ = Eologa - J] ^-^((C„)^>^. (2.23) 

ra>l 

Combining (2.21), (2.22) and (2.23) we see that (2.20) can be written as 

_«V^t:^E(i?^-pB„Cri + (p-l)(C„)^) <0 (2.24) 

which holds true using condition (1.11) and the fact that x^ — pxy^~^ + (p ^ l)y^ > for all 
x, y G M for even p > 2. This finishes the proof of the upper bound. 

D 

Before proving the invariance properties of Theorem 1 let us finish the proof of Theorem 
2 by showing that for invariant measures J^mv the upper bound decouples. 
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Lemma 7 For all fi G Aiinv, 'PnifJ') = "Pif^) for all n > 1. 

Proof. If we recall Ai defined in (1.27) then we can rewrite (2.14) as 

1 1 
P„(/i) = log 2 + - E log E' exp V Ai E log E' exp V Bi. (2.25) 



n ^ — ' n 

i<n i<n 



The result will follow if we show that for any n > 1, 

E' exp y^i^„ , 1 A 
E log ^^^"+^, = E log E' exp A^^, (2.26) 

and 

^^''^^^W^^^ = ^l«g^'^-P^n+i. (2.27) 

ic. exp 2^j<„ -Dj 

To prove this we will use the invariance properties (1.28) and (1.29). If in (1.29) we choose 
r to be a Poisson r.v. with mean n{p — l)a then it becomes 

We will only show how (1.28) implies (2.26) since the proof that (2.28) implies (2.27) is 
exactly the same. We only need to prove (2.26) conditionally on the Poisson r.v. 7r„+i(pa) 
and functions {6k,n+i) in the definition of At+i, 

exp An+i = Avg exp ^ Ok,i{e, Si,„+i,fc, . . . , Sp-i,„+i,fc), (2.29) 

k<TTn+l{pa) 

since we can control these functions uniformly with high probability using condition (1.13). 

Approximating the logarithm by polynomials, in order to prove (2.26) it is enough to prove 

that 

'E' exp An+i exp Y,.^^^ Ai n g 



for all q > I. Condition (1.10) implies that the right hand side of (2.29) is a polynomial of 
spins {sj^n+i,k) for k < 7in+i{pa) and j < p—l and, therefore, (2.30) is obviously implied by 
(1.28) if we simply enumerate spins (sj>+i,fc) as spins (sj) for n + 1 < i < -m by choosing m 
large enough. Averaging over random 7r„+i(pa) and (^fc,n+i) proves (2.30) and finishes the 
proof. 

Let us note that, similarly, (1.29) implies 

E'exp V.^„, , ^j(si j, . . . , Sr, j) 

log -^ — — — = ElogE exp6'i(si,i,...,Sp,i). 

E'exp)_^.<„^i(si,i, ...,Sp,i) 

which obviously implies (1.26), i.e. 

ElogE'expE = E log E' exp ^ 6*^(51,^, . . . , Sp,fc) 

fc<7r((p— l)a) 

= (p — l)Q;ElogE'exp^(si, . . . ,Sp). 
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2.4 Invariance and self-consistency equations. 

Proof of Theorem 1. Let h = YIkq^i where hi = YljeCi'^j- Consider fi E Ai which is 
a hmit of /in over some subsequence (iV^). Using Lemma 3, the left hand side of (1.23) 
is the hmit of E(/i)^ over subsequence (Nk). The right hand side of (1.23) wiU appear 
as a similar limit once we rewrite this joint moment of spins using cavity coordinates and 
"borrowing" some terms in the Gibbs measure from the Hamiltonian (1.15). The spins with 
coordinates i < n will play the role of cavity coordinates. Let us separate the TT{a{N + n)) 
terms in the first sum 

J2 Ua,^^^,...,a.^J (2.31) 

k<-K(a{N+n)) 

in (1.15) in the Hamiltonian H^+n into three groups: 

(1) terms for k such that all indices ii^fc, . . . iip,k > n; 
For 1 < j < n : 

(2j) terms with exactly one of indices ii^k, ■ ■ ■ , ip,k equal to j and all others > n; 

(3) terms with at least two of indices ii^k) ■ ■ ■ , ip,k < n. 
The probabilities that a term is of these three type are 

N \p _ I / N y-i _ v^ 



Pi = [ -^T— I ' P2,j = P ^r , [ -^T— I , P3 = i - Pi - 2^P2,l- 



Kn 



Therefore, the number of terms in these groups are independent Poisson random variables 
with means 

a{N + n)pi = a{N + n-np) + 0{N^'^), a{N + n)p2,j = ap + 0(A^"^), a{N + n)p3 = 0(A^"^). 

We can redefine the number of terms in each group to be exactly of means a{N + n — np), ap 
and since asymptotically it does not affect E,{h)N+n- Thus, if we write cr = {e, p) G T^N+n 
for the first the first n cavity coordinates e = {ei, . . . ,e„) and the last A^ coordinates p = 
(pi, . . . , Pjv) then (2.31) can be replaced with 

k<TT{a{N+n—np)) j<n k<TTj(ap) 

where indices ii^^, . . . , ip^k and ii^kj-, ■ ■ ■ ■, ip-i,k,j are all uniformly distributed on {1, . . . , A^}. 
Let us now consider the perturbation term in (1.15) 

Y^ logAv.exp ^ 4,K^,o^ii.M'---'^ip-i,fcJ (2-33) 

«<7r(cjv+n) k<i!i{ap) 

where ji^k^h ■ ■ ■ ,jp-i,k,i are uniformly distributed on {1, . . . , iV + n}. Here, we used indepen- 
dent copies 7ti and 9k^i since tij and 9k j were already used in (2.32). The expected number 
of these indices that belong to {1, . . . , n} is c^^nO^pip ~ l)n/N — )■ which means that with 
high probability all indices belong to {n + 1, . . . , N + n}. As a result, asymptotically K{h)N^n 
will not be affected if we replace the perturbation term (2.33) with 

J2 logAv.exp J2 (^k,ii^^Pn,k,n---^Pj,-i,kJ (2-34) 

Z<7r(cjv+„) k<TTi{ap) 
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where ji,k,i, ■ ■ ■ ,jp-i,k,i are uniformly distributed on {1, ... , N}. Thus, we can assume from 
now on that E(/i)jv+n is computed with respect to the Hamiltonian which is the sum of (2.32) 
and (2.34). If (■)'^ denotes the Gibbs average on S^f with respect to the Hamiltonian 

k<w{a(N+n—np)) 



then we can write 



Z<7r(cjv+n) 


k<Tti(ap) 


^{h)N+n 





(2.36) 



where 

Un,i = [Ave hi{e, p) exp ^ ^ dkjisj, p^,,,,, . • . , pi^_. 



j<n k<-Kj{ap) 



and 

Vn = (Av, exp J2 Yl ^^'j(^J' ^^i.fe.. ' • • • ' ^^p-i.^.. • / ^ 

j<n k<nj(ap) 

Finally, given r > 1, let us borrow r terms from the first sum in (2.35) by splitting the last 
r terms and replacing the first sum in (2.35) with 

k<Tr{a{N+n—np))~r k<r 

Here we ignore the negligible event when 7r(a(A^ + n — np)) < r. If we define 

-Kip) = Y ^^K.,---,P...J (2.37) 

k<Tr(a(N+n—np))—r 

+ Y log Av, exp Y ^M(^.Pii.M'---'%-i.M) 

l<n(cN+„) k<ni(ap) 

and let {■)'l^ denote the Gibbs average on Ejv with respect to this Hamiltonian then U^^i/Vn = 
Un,i/Vn where 

Un,i = (Av, hi{e, p) exp Y Y ^^J^^^' ^^i.^.i' • • • ' /^Vl^ J ^^P Y ^k{ph,k^- ■ ■ ^ Po,,S) ^ 

j<n k<iTj{ap) k<r 



Vn = (Av, exp Y Y ^^'^^^J' ^h.^,,- • • • > P*p-i.fc.,) exp Y ^k{Pn,„ ■■■, Pj, 



and 

j<n k<-Kj{ap) k<r 

By Lemma 2, the distribution of spins under the annealed Gibbs measure IE(-)jv correspond- 
ing to the Hamiltonian H'l^{p) still converges to p over the subsequence (iV^). Conditionally 
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on {-Kji^ap)), {Ok,j), (Ok) and on the event that all indices ii^kj, ■ ■ ■ , h-i,k,j and ji^k, ■ ■ ■ ,jp,k are 
different, Lemma 1 implies that the right hand side of (2.36) converges over subsequence (iV^) 
to ^YlKaUi/V^ where (f//) and V are defined in (1.21) and (1.22) only now conditionally 
on the above sequences. Since asymptotically all indices are different with high probability, 
the same convergence holds unconditionally and this finishes the proof. 



3 Sherrington-Kirkpatrick model. 

3.1 Properties of convergence. 

Of course. Lemma 1 still holds since it does not really depend on the model. However, the 
role of this lemma in the Sherrington-Kirkpatrick model will be played by the statement 
that we made at the beginning of the introduction which we now record for the reference. 

Lemma 8 The joint distribution of spins (a') and multi- overlaps (1.6) converges to the 
joint distribution of spins (1.3) and multi- overlaps (1.7) over any subsequence along which 
Hn converges to jj,. 

Lemma 2 also has a straightforward analogue for the Sherrington-Kirkpatrick model. Let (■) 
denote the Gibbs average with respect to the sum of an arbitrary Hamiltonian on Sjv and 
a perturbation term (1.35) and let (■)' denote the Gibbs average corresponding to the sum 
of the same arbitrary Hamiltonian and a perturbation as in (1.35) only with the number of 
terms replaced by ti{cn) + n instead of 7r(cAr) in the first sum and ti\cn) + m instead of 
7r'(cAr) in the second sum, for any finite m,n> 1. Then the following holds. 

Lemma 9 For any bounded function h of finitely many spins, or finitely many multi- overlaps, 
we have 

lim \E{hy -E{h)\ =0. (3.1) 

The proof is exactly the same as in Lemma 2. The role of the perturbation (1.35) will finally 
start becoming clear in the following exact analogue of Lemma 3. 

Lemma 10 // fiN converges to fi over subsequence (Nk) then it also converges to fi over 
subsequence {N^ + n) for any n > 1. 

Proof. We will show that the joint moments of spins converge to the same limit over sub- 
sequences that differ by a finite shift n. Let h = YljKq^j where hj = Yliec '^l '-'^^^ some 
finite sets of spin coordinates Cj. Let us denote by {■)n the Gibbs average with respect to 
the Hamiltonian (1.15) defined on N coordinates. We will show that 

lim |E(/i)^+„-E(/i)jv| = 0. 

Af^oo 
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Let us rewrite K{h)N^n by treating the last n coordinates as cavity coordinates. Let us 
write (T = (p, e) G Sjv+n for the first A^ coordinates p = (pi, . . . , ptv) and the last n cavity 
coordinates e = {ei, . . . ,e„) and rewrite (1.31) as 

-Hr,+nip) + Yl ^'^^^(P) + ^('^) (3-2) 

i<n 

where 

-Hn+tXp) = Yl fjv + nVp-i)/2 5Z 9iu...,i,Pii ---Pi,, (3.3) 

p>l ^ ' l<ii,...,ip<N 

the term eiZi{p) consists of all terms in (1.31) with only one factor Ei from e present, and 
the last term 6 is the sum of terms with at least two factors in e. It is easy to check that 

EZ,(pi)Z,(p2) = e{R{p\ P')) + ojv(l) 

uniformly over all p^,p^ and the covariance of S{cr) is also of small order uniformly over 
(T^,(T^. By the usual Gaussian interpolation one can therefore redefine the Hamiltonian 

HN+nio-) by 

-HN+ni(T) = -HN+nip) + J] ^^^^(P) (3-4) 

i<n 

where Gaussian processes Zi{p) have covariance C,'{R{p^, p^))- We can replace the perturba- 
tion term —H^^^{(t) by 

-Hf,{p)= Y logchGe,fe(p)+ Y. ^^'^^P^ (3.5) 

without affecting K{h)N+n asymptotically, since by Lemma 9 we can slightly modify the 
Poisson number of terms using that \cN+n — Cn\ — )■ and then replace G^',j(cr) and Gg^i{(7) 
by G^/^i{p) and Gg^i{p) by interpolation using that cat = o{N). If (■)^ denotes the Gibbs 
average on Sjv with respect to the Hamiltonian 

-H'^ip) = -H^Up) - Kip) (3.6) 

then each factor in 



{h)N+n = ll(h^)N^n = 11(11 ^^) N^n = UiU p) 

can be written as 



N+n 
j<q j<q ieCj j<q i£Cj 



{hj)N+n - — =^ ., - \11 Pi/N 

(Av,exp^.<„5,Z,(p))^ iec,- 

where (■)^ is the Gibbs average on Sjv corresponding to the Hamiltonian 

-ij;^(p) = -/j;,(p) + 5^iogchz,(p). 



i<n 
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Thus, K{h)N+n = ^{h)%. Since Zi{p) are independent copies of G^/{p), in distribution 

where 

-H^/{p)= Yl logchG^,,,(p)+ Yl ^^^^^Py (3.7) 

fc<7r(c]v)+« fc<7r'(c]v) 

Let us now consider E,{h)N- It is easy to check that, in distribution, the Hamiltonian Hn{p) 
can be related to the Hamiltonian Hjy+nip) in (3.3) by 

-Hm{p) = -HN+n{p) + 5^ V;(p) (3.8) 

i<n 

where {Yi{p)) are independent Gaussian processes with covariance 

Ey,(pi)F,(p2) = e{Rip\ p2)) + o^(i). 

Again, without affecting E,{h)N asymptotically, one can assume that the covariance of Yi{p) 
is exactly 9{R{p^, p"^)) which means that they are independent copies of Go{p). Therefore, 
we can assume that K{h)N is taken with respect to the Hamiltonian 

-Kip) = -H^Up) - HfiP) 

where 

-Hf{p)= J2 logchG5,,,(p)+ Y ^<^^^^P)- (3.9) 

fc<7r(c]v) fc<7r'(c]v)+" 

Lemma 9 then implies that both perturbation terms (3.7) and (3.9) can be replaced by the 
original perturbation term (1.35) without affecting IE(/i)jv ^"^^ ^{h)N asymptotically and 
this finishes the proof. 



3.2 Lower bound. 

Lemma 11 There exists ^ E Ai such that lim^v-^oo -^Af ^ "^(z^)- 

Proof. We again use (2.9). Suppose that this lower limit is achieved over subsequence (iV/j) 
and let /i G A^ be a limit of (/xat) over some subsubsequence of (iV^). Let Z^ and (■) be 
the partition function and the Gibbs average on Sjv corresponding to the Hamiltonian iJ^ 
defined in (3.6) and let us compute the limit of 

Ti?i Zn+1 „, Zn 
Elog^ Elog-^ 

along the above subsubsequence. Using (3.4) and (3.8) for n = 1 and the fact that, as in 
(3.5), the perturbation Hamiltonian Hff_^^[(T) in Zat+i can be replaced by H^[p), the above 
limit is equal to the limit of 

log2 + Elog(chG5,(p))-Elog(expGe(p)). 
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It remains to show that 



hm E\og{chG.'{p)) =E\ogE'chG.'{aJw,u,-)) (3.10) 

AT— >-oo 



and 



hm Elog{exp Gg{p)) =E\ogE'expGg{aJw,u, ■)) (3.11) 

TV— i-oo 

where for simphcity of notations we wiU write hmits for iV — )■ oo rather than over the above 
subsubsequence. The proof of this is identical to Talagrand's proof of the Baffioni-Rosati 
theorem in [28]. First of aU, if Eg denotes the expectation in the randomness of G^i{p) 
conditionally on the randomness in (■) then standard Gaussian concentration implies that 
(see, for example, Lemma 3 in [20]) 

P3(|log(ch G'e(p)) -E,log(chG'e(p))| >A)< e-^^' 
for some small enough constant c and since 

< E,log(chG'e(p)) < log(E,chG'e(p)) < e'(l)/2 
for large enough A > we get 

P(|log(chG^,(p))|>A)<e-^^^ (3.12) 

Therefore, if we denote log^ x = max(— A, min(loga;. A)) then for large enough A, 

|Elog(chG'^,(p)) -Elog^(chG'^,(p))| < e""^'. (3.13) 

Next, if we define ch^x = min(chx, chA) then using that 

I log^x — log^yl < e |a; — yl and |cha; — ch^x| < chx J(|x| > A) 
we can write 

|Elog^(chG'e(p))-Elog^(chAG'e(p))| < e^ E( I chG'e(p)-ch^ 6-^(^)1) 

< e^E{chG^ip)Ii\G^ip)\>A)). 

By Holder's inequality we can bound this by 

e^ {E{Egch'G^ip))Y^' (E(P,(|G^,(p)| > A))f' < e~^^' 

for large enough A since ¥g{\G^i{p)\ > A) < e^'^^ . Combining with (3.13) proves that 

|Elog(chG'^,(p)) -Elog^(ch^G'^,(p))| < e-^^'. (3.14) 

Approximating logarithm by polynomials on the interval [e""^, e"^] we can approximate 
Elog^(ch^Gg/(p)) by some linear combinations of the moments 

E(ch^ G^>{p)Y = E( J] ch^ G^.{p')) = E(E, J] ch^ G^,{p')) 

l<q l<q 
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for q > 1. Since 

E, J]ch^G'5,(p') = F((i?;,,OM'<.) (3-15) 

l<q 

for some continuous bounded function F of the overlaps {Ri,i')i,i'<q, Lemma 8 implies that 



lim E(E,TTch^G'e(p')) =EF((i?;^,)M'<.) 

N—^OD -*- -*- ' 

l<q 



Let us rewrite the right hand side in terms of the process G^i in (1.38). Recall the definition 
of the processes in (L38) and (1.39). If E^ is the expectation in the Gaussian randomness 
of these processes then the definition of the function F in (3.15) implies that 

EF((i?;7,)/,/'<g) = EEg n chA G^ia^iw, m, ■)) = E(E'chA Gi,,{<y^{w, u, -W 

l<q 



and, therefore, 



lim Elog^(chAG'^'(p)) = Elog^E'chAG'5'(o-^(t(7,M, ■)). 

TV— >-oo 



(Notice that this approximation by moments depended on functions of the overlaps only 
which justifies the comment leading to the equation (1.52).) One can show similarly to 
(3.14) that 

\ElogE'chG^'{a^{w,u,-))-'^logj^E'chAG^>{a^{w,u,-))\ < e~'^' (3.16) 

which finishes the proof of (3.10). Equation (3.11) is proved similarly. 



3.3 Upper bound and free energy. 

Since the perturbation term in (1.15) does not affect the limit of free energy, we will now 
ignore it and consider free energy F/v defined for the original unperturbed Hamiltonian (1.31). 

Lemma 12 For any function a : [0, 1]^ — ?> [—1, +1] we have 
Fn <log2 + -ElogE' Y[chG^,,,{cT{w,u,-)) - -ElogE' expJ2GeAHw,u,-)). (3.17) 

i<N i<N 

Proof. This is proved by the Guerra type interpolation as in [13]. If for t G [0, 1] we consider 
the interpolating Hamiltonian 

-HN,t{cr) = -ViHN{cr) + Vl - t y^ aiG^>Aa{w, u, ■)) + Vi^Ge,iid-{w,u,-)) 

i<N i<N 

and interpolating free energy 



cfit) = ^ElogE' 5^ expi-Hj,,ticT)) 



N 
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then to prove (3.17) it is enough to show that (p'{t) < 0. This is done by the usual Gaussian 
integration by parts as in [13]. 

D 

Before proving invariance properties of Theorem 3 let us finish the proof of Theorem 4 
by showing that if we let 

n -'■-'■ n ^-^ 

i<n i<.n 

then the invariance of Theorem 3 implies the following. 

Lemma 13 For all ji G A^m«, Vn{lj) = V^ji) for all n > 1. 

Proof. The result will follow if we show that for (J = a^ for any n > 1, 

Elog 'n-"^\^^'V \ =ElogrchG^>+i(a(t.,t.,-)) (3.18) 

and 

E' exp V .^„ , -, Ge i(a(w, -u, ■)) 
Elog ^^^"+^. ;_/ \^ =E\ogE'expGe,n+i{<y{w,u,-)). (3.19) 

E exp)^.<„G'0,i(CT(u;,M, ■)) 

To prove this we will use invariance properties (1.45) and (1.46). Using truncation and 
Gaussian concentration as in Lemma 11, to prove (3.18) it is enough to show that 

E — pf -, TT 1 =E(Ech^G'^,,„+i a w,M,- ) • 

Using replicas as in (1.48), the left hand side can be written as 

E^^nK,n.<nChGe.(a(^,^z,-)) 

(E'ri.<„chG^,,(a(^, «,■)))« ^ • ^ 

where 

F = F((i?,70M'<9) = Eg n cl^^ G^,,n+i{cx{w, m, ■)) 

l<q 

is a bounded continuous function of the overlaps defined in (1.8). Approximating F by 
polynomials of overlaps and using (1.45) proves that (3.20) is equal to 

EF = EEg n chA G'e,n+i(^(tt', ui, ■)) = E(rch^ G'^,,„+i(a(w, u, ■)))' 

l<q 

and this finishes the proof of (3.18). Equation (3.19) is proved similarly using (1.46) instead. 
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3.4 Invariance and self-consistency equations. 

Proof of Theorems 3 and 5. Let h = YIkq ^i where hi = Yliec ^l- Consider fj, E Ai which 
is a hmit oi hn over some subsequence {Nk). By Lemma 10, the left hand side of (1.42) is the 
hmit of E,{h)N+n over subsequence {Nk). The right hand side of (1.42) will appear as a similar 
limit once we rewrite this joint moment of spins using cavity coordinates. The beginning of 
the proof will be identical to the proof of Lemma 10, only the spins with coordinates i < n will 
now play the role of cavity coordinates instead of spins with coordinates N +1 <i < N + n. 
Let us write a = (e, p) G SAr+„ for the first n cavity coordinates e = (ei, . . . , e„) and and 
the last A^ coordinates p = (pi, . . . ,Pn)- Let us consider sequences of Gaussian processes 
{Zi[p)) and {Yi{p)) which are independent copies oi G^'{p) and Gg{p) correspondingly. First 
of all, we can replace the perturbation term —H'^j^^{a) with 

-Hl{p)= J2 logchG5,,fc(p)+ J2 GeAp) + J2^k{p) (3.21) 

fc<7r(cjv) fc<7r'(cjv) k<r 

without affecting E(/i) asymptotically, since by Lemma 9 we can slightly modify the Pois- 
son number of terms and then we can replace G^/^j(cr) and Gg j(cr) with G^/ j(p) and Gg^i{p) 
by interpolation using that c^ = o{N). Then, as in (3.4), we can redefine the Hamiltonian 
-HN+n{cr) by 

-HN+n{0-) = -HN+nip) + J2 ^'^'^P^ (3.22) 

i<.n 

where HN+n{p) is defined in (3.3). Let (■) denote the Gibbs average corresponding to the 
Hamiltonian 

-H'^{p) = -Hr,+n{p)+ Yl ^ogchG^,4p)+ Y, GeAp)- (3.23) 

fc<7r(c]v) k<n'(cff) 

Recalling the relationship (3.8) between H^{p) and H^^n{p), let us note that Lemma 9 
implies, as in the proof of Lemma 10, that the joint distribution of spins /x^ corresponding 
to the Hamiltonian (3.23) converges to the same limits (over subsequences) as the original 
sequence ^n- Let us write the function hi{(T) in terms of e and p as 

hiio-) = n cTi = n ^* n "^^ ^ n ^* n '^^ 

where we will abuse the notations and still write Gf to denote the set of coordinates pi 
corresponding to the original coordinates 0"„+j. Then we can write 



V? 



E(/i)^+„ = Eii%^- (3.24) 



' N 



where 



Un,i = (Avs Y[ SiSxpY^i^iiP) n Piexp^Ffc(p) 

ieCl i<n ieCf k<r 
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and 

Vn = (^Av^exp^€iZi{p)exp^Yk{p)j = (expX(p)) 

i<n k<r 

where we introduced 

X(p) = ^logchZ,(p) + 5^n(p). 

i<n fc<r 

It remains to show that 

j.^^rw^^^rw^ (3.25) 

where 

Ui = E'Ave Y\ ^i 6xp ^ ^^^^'.^(^(w, u, ■)) J]^ CTi exp ^ ^^^^(^(w, M, ■)) 

ieC/ i<n i(^Cf k<r 

for cTj = d-{w,u,Vi) and 

y = E'Ave exp ^ ^^^^/^^(^(m;, m, ■)) exp ^ ^^^^(^(w, m, ■)) 

i<n k<r 

which is, of course, the same equation as (1.42). (The proof that (1.58) imphes (1.62) is 
exactly the same of the proof of (3.25).) The proof of (3.25) is nearly identical to the proof 
of (3.10) using truncation and Gaussian concentration, only instead of approximating a 
truncated version of log x by polynomials we now need to approximate truncated version of 
1/x by polynomials. If we denote 

Y = log Vat = log(expX(p)) 

then, as in (3.12), one can show that for large enough A> 

mY\ >A)< e-^^\ (3.26) 

For A > let {x)a = max(— A, min(x, A)) so that 

I exp(— gx) — exp(— g(a;)A)| < max(e~'^'^,exp(— ga;))J(|a;| > A). 

If we denote Z = YIko Un,i then, obviously, EZ^ < L for some large enough L > that 
depends on q,n,r and function ^, and (3.26) implies that 

|EZexp(-gr) -EZexp(-g(r)A)| < E\Z\ max(e"«^,exp(-gr))/(|r| > A) < e"^^' (3.27) 

for large enough A. Next, let exp^x = max(e~^,min(expx, e"^)) and let Y' = log(exp^X(p)). 
Since for all x, t/ e M 

I exp(— g(x)^) — exp(— g(y)^)| < qe^'^^^''^\ expx — expy\ 

we get 

|exp(-g(r)^) -exp(-g(r')A)| < ge^^+^^^d expX(p) - exp^X(p)|). 
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Next, since for all a; G M 

I expx — exp^x| < max(e~ ,expa;) /(|a;| > A) 

we obtain the following bound 

|exp(-g(r)^-exp(-g(F')A)|<ge(''+i^^(max(e-'^,expX(p))/(|X(p)|>A)). 

It is easy to see that P(|X(p)| > A) < e~'^^ for large enough A and using Holder's inequality 

\EZ exp{-q{Y)A) -EZ exp{-q{Y')A)\ < ge('?+i)^(EZ2)i/2(E(max(e-^^,exp4X(p))))i/^ 

x(E(/(|X(p)| > A)))i/^ < e"^^'. 

Combining this with (3.27) we proved that 

\EZ exp{-qY) -EZ exp{-q{Y')A)\ < e""^' 

for large enough A. We can now approximate exp(— ^(y)^) = (exp^X(p))^^ uniformly by 
polynomials of (exp^X(p)) and therefore EZexp(— ^(y)^) can be approximated by a linear 
combination of terms 

El[U^^i{expAXip)y. (3.28) 

l<q 

If we write the product of the Gibbs averages using replicas and take expectation with respect 
to the Gaussian processes (Xj(p)) and {Yi{p)) inside the Gibbs average we will get a Gibbs 
average of some bounded continuous function of finitely many overlaps in addition to the 
spin terms HiGC^ A ^^^^ appear in the definition of U^^i. Observe that if from the beginning 
we chose m = n then factors IIjgc^ Pi would not be present, which means that the linear 
combination of (3.28) gives an approximation of E(/i)Ar+„ (and thus E(/i)7v) by the annealed 
Gibbs average of some functions of overlaps only. In particular, this proves Theorem 5. In 
the general case. Lemma 8 implies that (3.28) converges to E]^^^ ///(V^)^ where 

Va = E' exp^ (^ log ch G^'^i{a{w, u, ■)) + ^ Ge^k{^{w, u, ■)) j . 

i<n k<r 

Since the same truncation and approximation arguments can be carried out in parallel for 
the right hand side of (3.25) this proves (3.25) and finishes the proof of Theorem 1.42. 

D 

Proof of Theorem 7. Using Gaussian integration by parts and invariance in (1.62), 
(1.65) can be rewritten as 

1 - t^{E{R^^fP - 2E(i?~ )*'(i?~3)P + (E{R'^^yf) 

and the second term disappears whenever the Ghirlanda-Guerra identities (1.59) hold. On 
the other hand, if (1.65) is uniformly bounded for all t > then for any bounded continuous 
function F of multi-overlaps (1.7) on n replicas 

^ FGpiajw, ui, ■)) exp t ^^^^ Gp{cx{w, m, ■)) ^^^ Gp{a{w, u, ■)) exptCpjajw, u, ■)) 

(E'exptG'p(d-(w,M, ■)))" E'exptGp{a{w,u,-)) ^ ' ' 
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is also uniformly bounded by invariance in (1.62) and Holder's inequality. Using Gaussian 
integration by parts and invariance in (1.62), this is equal to 

n 

t {Y^ EFiR^iY - nEF{R^^^-^y + EF E(i?~ )^) (3.30) 

1=2 

which can be bounded only if (1.59) holds. 
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